The present disclosure relates to an image processing apparatus, an image processing method, and a program. More particularly, the present disclosure relates to an image processing apparatus, an image processing method, and a program, which remove noise included in an image.
As technology for removing noise from an image, a process of filtering the image is well known. For example, various techniques including traditional image processing technology as in a median filter or a Wiener filter have been proposed
Recently, a widely used technique has been a bilateral filter or a non-local (NL)-means filter.
The bilateral filter is disclosed, for example, in C. Tomasi and R. Manduchi's “Bilateral Filtering for Gray and Color Images,” Proceedings of the IEEE International Conference on Computer Vision, 1998.
The NL-means filter is disclosed, for example, in A. Buades, B. Coll, and J. M. Morel's “A Non-Local Algorithm for Image Denoising,” Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, 2005.
The filter application processes are processes of correcting a pixel value of a pixel of interest, for example, by designating peripheral pixels around the pixel of interest serving as a denoising target as reference pixels and employing an arithmetic mean value of a plurality of pixel values of the reference pixels in any technique.
In the technique of the related art, the peripheral pixels are usually selected from a rectangle region centered on a pixel-of-interest position. If a filtering operation is performed using a signal of an isotropic region, a phase shift cannot easily occur in the processed image and a visually desirable image can be obtained.
On the other hand, a general camera signal processing section, that is, a camera signal processing section for processing an output signal of an imaging element, has a kind of memory referred to as a delay line on which pixel values of row units previously read from the imaging element are stored, and generally performs various signal processing operations such as pixel value correction while referring to pixel data of a predetermined local region including memory-stored data.
A configuration in which means for executing an “edge-preserving smoothing process” similar to the above-described bilateral filter with a delay line used in many cameras is mounted as a denoising means has been proposed.
In principle, the “edge-preserving smoothing process” is a low pass filter (LPF) application process. The strength of smoothing can be controlled by a cutoff frequency of the LPF.
Although the strength of smoothing can be increased by decreasing a cutoff frequency, a pixel value of a position farther from a pixel of interest should be referred to for implementation of a filter of which the cutoff frequency is low. In order to refer to distant pixels in camera signal processing, pixel values of pixels apart from the pixel of interest need to be retained and consequently, a memory of more delay lines is necessary.
That is, there is a problem in that hardware cost is necessarily increased to implement a high-performance denoising process.
In order to solve the problem that the hardware cost is increased when a broad region is referred to, various methods using an infinite impulse response (IIR) filter have been proposed, for example, in Japanese Patent Application Laid-Open Nos. 2001-36769, 2005-184786, 2005-286678, and 2010-81497. In the IIR filter, a filtering operation is performed as a process only referring to a current signal and a previous signal with respect to a signal input along time series.
In processing according to an embodiment of the present disclosure, description will be given using a primary IIR filter. When the IIR filter is used in image processing, the processing is performed using pixels of an anisotropic region with respect to a pixel of interest.
Thus, there is a problem in that a phase is easily shifted and an edge of an image is blurred in a specific direction. A method of weakening the strength of smoothing is effective as a method of suppressing the phase shift. This is implemented in any method in the related art.
However, there is a problem in that control of the strength of smoothing is insufficient in the related art. In principle, an edge-preserving smoothing process based on the IIR filter calculates a low frequency component of a signal using the IIR filter and blends an original signal according to edge strength of the original signal.
An edge is preserved by further blending the original signal when there is a clear edge in a signal.
On the other hand, by blending the original signal as little as possible when there is no clear edge, only a low frequency component of a signal is obtained and smoothing is performed.
It is possible to selectively suppress only small amplitude (=noise) in a flat part of a signal by performing the process as described above.
More specifically, the edge-preserving smoothing process based on the IIR filter will be described with reference to the following Expression (1).pIIR(t)=α(t)p(t)+(1−α(t))pIIR(t−1)  (1)
In the above-described Expression (1), t denotes a time, p(t) denotes a signal value (=a pixel value before correction of a pixel of interest of a correction (denoising) target), and pIIR(t) denotes an edge-preserving smoothed signal (=a pixel value after the correction of the pixel of interest) using the IIR filter.
α(t) is a variable of which a range is [0, 1], that is, α=0 to 1, and is adaptively changed so that the more the strength of an edge, the greater the value.
It is preferable that the variable α(t) continuously vary with the edge strength. This is because a discontinuous change in edge detection results in an extreme change of the smoothing strength and is visually undesirable.
In the related art, there is a problem in that sufficient performance is not obtained or a discontinuous change in the edge detection leads to the degradation of image quality because a method of determining the variable α(t) is ad hoc.
In Japanese Patent Application Laid-Open Nos. 2001-36769 and 2005-184786 described above, there is a problem in that switching of processing is conspicuous because the presence/absence of an edge is classified into two values in the edge detection.
In addition, although the edge-preserving smoothing process using a method corresponding to the above-described Expression (1) is performed in Japanese Patent Application Laid-Open Nos. 2005-286678 and 2010-81497, there is a problem in that the edge is not necessarily sufficiently preserved because an arbitrarily determined broken line is used when the variable α(t) is determined.